Analysis of Crowdsourced Sampling Strategies for HodgeRank with Sparse Random Graphs

نویسندگان

  • Braxton Osting
  • Jiechao Xiong
  • Qianqian Xu
  • Yuan Yao
چکیده

Crowdsourcing platforms are now extensively used for conducting subjective pairwise comparison studies. In this setting, a pairwise comparison dataset is typically gathered via random sampling, either with or without replacement. In this paper, we use tools from random graph theory to analyze these two random sampling methods for the HodgeRank estimator. Using the Fiedler value of the graph as a measurement for estimator stability (informativeness), we provide a new estimate of the Fiedler value for these two random graph models. In the asymptotic limit as the number of vertices tends to infinity, we prove the validity of the estimate. Based on our findings, for a small number of items to be compared, we recommend a two-stage sampling strategy where a greedy sampling method is used initially and random sampling without replacement is used in the second stage. When a large number of items is to be compared, we recommend random sampling with replacement as this is computationally inexpensive and trivially parallelizable. Experiments on synthetic and real-world datasets support our analysis.

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

HodgeRank with Information Maximization for Crowdsourced Pairwise Ranking Aggregation

Recently, crowdsourcing has emerged as an effective paradigm for human-powered large scale problem solving in various domains. However, task requester usually has a limited amount of budget, thus it is desirable to have a policy to wisely allocate the budget to achieve better quality. In this paper, we study the principle of information maximization for active sampling strategies in the framewo...

متن کامل

Sampling in Potts Model on Sparse Random Graphs

We study the problem of sampling almost uniform proper q-colorings in sparse Erdős-Rényi random graphs G(n, d/n), a research initiated by Dyer, Flaxman, Frieze and Vigoda [2]. We obtain a fully polynomial time almost uniform sampler (FPAUS) for the problem provided q > 3d + 4, improving the current best bound q > 5.5d [6]. Our sampling algorithm works for more generalized models and broader fam...

متن کامل

On construction and analysis of sparse random matrices and expander graphs with applications to compressed sensing

We revisit the probabilistic construction of sparse random matrices where each column has a fixed number of nonzeros whose row indices are drawn uniformly at random. These matrices have a one-to-one correspondence with the adjacency matrices of lossless expander graphs. We present tail bounds on the probability that the cardinality of the set of neighbors for these graphs will be less than the ...

متن کامل

Construction and analysis of sparse random matrices and expander graphs with applications to compressed sensing

We consider a probabilistic construction of sparse random matrices where each column has a fixed number of nonzeros whose row indices are drawn uniformly at random. These matrices have a one-to-one correspondence with the adjacency matrices of lossless expander graphs. We present tail bounds on the probability that the cardinality of the set of neighbors for these graphs will be less than the e...

متن کامل

Weighted sparse recovery with expanders

We derived the first sparse recovery guarantees for weighted l1 minimization with sparse random matrices and the class of weighted sparse signals, using a weighted versions of the null space property to derive these guarantees. These sparse matrices from expender graphs can be applied very fast and have other better computational complexities than their dense counterparts. In addition we show t...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

عنوان ژورنال:
  • CoRR

دوره abs/1503.00164  شماره 

صفحات  -

تاریخ انتشار 2015